Context-based Evaluation of MT Systems: Principles and Tools
نویسندگان
چکیده
This tutorial introduces a principled approach to the evaluation of MT systems in their context of use. A method for defining contextual evaluation plans will be described, along with an interactive tool supporting the definition of such plans. This tutorial provides the audience with knowledge and tools for MT evaluation, which will enable them to better justify their choices of particular MT systems, or to argue about the quality of MT systems in a more principled way.
منابع مشابه
Evaluation of Endogenous Systems
In system development we are faced with the necessity of evaluation. Evaluation measures our success relative to other: a) theories of development or domains; b) implementations of similar theoretical principles; or c) increments of a given system. For successful software engineering evaluation, progress is measured against a model, a task frequently accomplished through requirements analysis. ...
متن کاملبهبود کیفی طراحی بر پایه مدیریت بهینه عوامل مؤثر
This paper investigates design factors and the interaction that occurs between the different elements of the design process. Here, at the beginning, the gap created between the level and the extent of capabilities formed in university environments was compared with capabilities expected from graduated people in their professional environments and In continuation, ignoring some parts of the re...
متن کاملContext-Aware Recommender Systems: A Review of the Structure Research
Recommender systems are a branch of retrieval systems and information matching, which through identifying the interests and requires of the user, help the users achieve the desired information or service through a massive selection of choices. In recent years, the recommender systems apply describing information in the terms of the user, such as location, time, and task, in order to produce re...
متن کاملExtending MT evaluation tools with translation complexity metrics
In this paper we report on the results of an experiment in designing resource-light metrics that predict the potential translation complexity of a text or a corpus of homogenous texts for state-ofthe-art MT systems. We show that the best prediction of translation complexity is given by the average number of syllables per word (ASW). The translation complexity metrics based on this parameter are...
متن کاملThe FEMTI guidelines for contextual MT evaluation: principles and resources
A large number of evaluation metrics exist for machine translation (MT) systems, but depending on the intended context of use of such a system, not all metrics are equally relevant. Based on the ISO/IEC 9126 and 14598 standards for software evaluation, the Framework for the Evaluation of Machine Translation in ISLE (FEMTI) provides guidelines for the selection of quality characteristics to be e...
متن کامل